Method and system for improved visibility in blended layers for high dynamic range displays

ABSTRACT

There are many instances where a standard dynamic range (“SDR”) overlay is displayed over high dynamic range (“HDR”) content on HDR displays. Because the overlay is SDR, the maximum brightness of the overlay is much lower than the maximum brightness of the HDR content, which can lead to the SDR elements being obscured if those elements have at least some transparency. The present disclosure provides techniques including modifying the luminance of either or both of the HDR and SDR content when an SDR layer with some transparency is displayed over HDR content. A variety of techniques are provided. In one example, a fixed adjustment is applied to pixels of one or both of the SDR layer and the HDR layer. The fixed adjustment comprises decreasing the luminance of the HDR layer and/or increasing the luminance of the SDR layer. In another example, a variable adjustment is applied.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/218,708 filed on Dec. 13, 2018, which is incorporated by referenceherein in its entirety.

BACKGROUND

High dynamic range (“HDR”) displays provide more pleasing visualexperiences by increasing the ratio between maximum and minimumbrightnesses as compared to standard dynamic range (“SDR”) displays.Improvements related to HDR displays are constantly being made.

BRIEF DESCRIPTION OF THE DRAWINGS

A more detailed understanding may be had from the following description,given by way of example in conjunction with the accompanying drawingswherein:

FIG. 1 is a block diagram of an example device in accordance withcertain implementations;

FIG. 2 is a block diagram illustrating aspects of the device of FIG. 1,according to an example;

FIG. 3 is a diagram illustrating an underlying content layer and anoverlay content layer, according to an example;

FIGS. 4A-4B illustrate comparative dynamic ranges of a display, anunderlying content layer, and an overlay content layer, according to anexample; and

FIG. 5 is a flow diagram of a method for applying adjustments to animage, according to an example.

DETAILED DESCRIPTION

Described herein are techniques for improving visibility in blendedlayers. Specifically, there are many instances where a standard dynamicrange (“SDR”) overlay is displayed over high dynamic range (“HDR”)content on HDR displays. In an example, HDR video content is provided toa cable television set-top box, which adds a guide and menus. Becausethe overlay is SDR, the maximum brightness of the overlay is much lowerthan the maximum brightness of the HDR content, which can lead to theSDR elements being obscured if those elements have at least sometransparency (e.g., an alpha value substantially lower than 1.0). Tohelp with this situation, the present disclosure provides techniquesincluding modifying the luminance of either or both of the HDR and SDRcontent when an SDR layer with some transparency is displayed over HDRcontent. A variety of techniques are provided. In one example, a fixedadjustment is applied to pixels of one or both of the SDR layer and theHDR layer. The fixed adjustment comprises decreasing the luminance ofthe HDR layer and/or increasing the luminance of the SDR layer. Inanother example, a variable adjustment is applied. The variableadjustment is based on a comparison between a characteristic value of aparticular pixel with a statistical result derived from multiple pixels.The term “characteristic value” refers to any value that characterizes apixel. Examples of characteristic values include alpha values, luminancevalues, average color values (or other functions of color values), orany other value that characterizes a pixel. In an example, a mean istaken of the alpha value of all pixels in the SDR layer. An adjustmentis made to each pixel, where the adjustment for a pixel is based on acomparison of the alpha value of the pixel to the mean. Many othertechniques are disclosed herein.

FIG. 1 is a block diagram of an example device 100 in which one or morefeatures of the disclosure can be implemented. The device 100 includes,for example, a computer, a gaming device, a handheld device, a set-topbox, a television, a mobile phone, or a tablet computer. The device 100includes a processor 102, a memory 104, a storage 106, one or more inputdevices 108, and one or more output devices 110. The device 100 alsooptionally includes an input driver 112 and an output driver 114. It isunderstood that the device 100 includes additional components not shownin FIG. 1.

In various alternatives, the processor 102 includes a central processingunit (CPU), a graphics processing unit (GPU), a CPU and GPU located onthe same die, or one or more processor cores, wherein each processorcore can be a CPU or a GPU. In various alternatives, the memory 104 islocated on the same die as the processor 102, or is located separatelyfrom the processor 102. The memory 104 includes a volatile ornon-volatile memory, for example, random access memory (RAM), dynamicRAM, or a cache.

The storage 106 includes a fixed or removable storage, for example, ahard disk drive, a solid state drive, an optical disk, or a flash drive.The input devices 108 include, without limitation, a keyboard, a keypad,a mouse, a touch screen, a touch pad, a detector, a microphone, anaccelerometer, a gyroscope, a biometric scanner, or a network connection(e.g., a wireless local area network card for transmission and/orreception of wireless IEEE 802 signals). The output devices 110 include,without limitation, a display device 118, a speaker, a printer, a hapticfeedback device, one or more lights, an antenna, or a network connection(e.g., a wireless local area network card for transmission and/orreception of wireless IEEE 802 signals).

The input driver 112 communicates with the processor 102 and the inputdevices 108, and permits the processor 102 to receive input from theinput devices 108. The output driver 114 communicates with the processor102 and the output devices 110, and permits the processor 102 to sendoutput to the output devices 110. It is noted that the input driver 112and the output driver 114 are optional components, and that the device100 will operate in the same manner if the input driver 112 and theoutput driver 114 are not present. The output driver 114 includes anaccelerated processing device (“APD”) 116 which is coupled to a displaydevice 118. The APD 116 is configured to accept compute commands andgraphics rendering commands from processor 102, to process those computeand graphics rendering commands, and to provide pixel output to displaydevice 118 for display.

FIG. 2 is a simplified block diagram of an example system 200, in whichone or more features of the disclosure can be implemented. System 200includes a processor 102 in communication with a rendering engine 210which collectively provide image data to a pre-display device 215. Theoptional pre-display device 215 reads the image data, optionallymodifies pixel data derived from the image data according to one or moretechniques, and outputs the modified pixel data to a display device 118for display. The system 200 represents one of a variety of devices thatcause graphics to be displayed on a screen.

In one example, the system 200 is a home computing device such as adesktop, laptop, tablet computer, smart phone, video game console, orother device. The processor 102 is a main processor (e.g., centralprocessing unit (“CPU”)) of the device. In such a system, the processor102 provides commands and data to the rendering engine 210. Therendering engine 210 represents dedicated or integrated graphicsrendering hardware. The rendering engine 210 generates pixel output fordisplay on the display device 118. The commands and data for therendering engine 210 instruct the rendering engine 210 to render framesbased on two-dimensional or three-dimensional objects. The commands anddata also specify an overlay for display on the display device 118,which the rendering engine 210 combines with the rendered frames togenerate a final output for display on the display device 118. In such asituation, no pre-display device 215 is present.

In another example where the system 200 is a home computing device, therendering engine 210 includes a video decoder that decodes video datafor display. The processor 102 issues commands to the video decoder todecode video data. A pre-display device 215 is a video post-processingengine that adds an overlay for display on the display device 118.

In another example, the system 200 is a video content server connectedover the internet and through the pre-display device 215 to the displaydevice 118. In such a system, the processor 102 and the rendering engine210 work in conjunction to provide pixel data for display on the displaydevice 118. The pre-display device 215 is a set-top box or other mediaplayer that receives the video data, adds an overlay, and outputs theresulting combined image frames to the display device 118 for display.

In another example, the system 200 is a compound system that includes adisplay system that is connected to a video content provider system thatprovides video content to the display system. In such an example, thedisplay system includes the display device 118 and the pre-displaydevice 215, whose role is to generate an overlay and composite theoverlay with video content received from the video content providersystem. Examples of the content provider system is a streaming desktopcomputer or an internet video content server. An example of the displaysystem is a smart television or a non-smart television that includes thepre-display device 215.

Although some examples of the system 200 are described, it should beunderstood that the system 200 may represent any other type of computingdevice that receives or generates underlying image content for displayand combines that underlying image content with an overlay for displayon a display device 118.

It is possible for the graphics of the overlay layer to be transparentlycomposited with the underlying content layer. A value, typicallyreferred to as “alpha,” specifies the opacity of the overlay layer. Inother words, the alpha value specifies the degree to which the color ofthe overlay layer contributes to final color value of a blended pixel.By convention, an alpha value of 1.0 indicates that the overlay layer isopaque and that therefore, the final color value would be the colorprovided by the overlay layer, which is unmodified by the underlyingcontent layer. An alpha value of 0.0 indicates that the overlay layer iscompletely transparent and makes no contribution to the final colorvalue. An alpha value of 0.5 indicates that both the underlying contentand the overlay layer contribute to a substantial degree to the finalpixel value.

Display devices like display device 118 have an inherent dynamic range.The dynamic range specifies the ratio between the highest brightness andthe lowest brightness that the display is capable of producing.Traditional displays are considered to have a “standard” dynamic range(“SDR”). Although the actual dynamic range of different displays varies,the output from typical applications is usually tone mapped to an SDRrange. Tone mapping is a technique for mapping the brightnesses ofpixels output by a content generator (e.g., by the rendering engine 210)to actual brightness values that are output by a display. Typically,tone mapping is done to approximate the intended appearance of an imagedespite the limitations of display.

High dynamic range (“HDR”) displays have the capability to display amuch greater range of brightnesses than SDR displays. In addition, themaximum brightness of HDR displays is much greater than the maximumbrightness of SDR displays (for example, 1000 nits for an HDR displayversus 300 nits for an SDR display). However, to use the full dynamicrange of HDR displays, content that is to be displayed on HDR displaysneeds to be designated as HDR content, which has a much greater dynamicrange than SDR content. SDR content displayed on an HDR display will notbe mapped to the full range of the display but will instead be mapped toa dimmer portion of the dynamic range of the HDR display. SDR content is“mapped down” in this manner because of the greater brightness of HDRdisplays as compared with SDR displays. More specifically, HDR contenttypically includes a “highlight range” that comprises the upper portionof the HDR range. This highlight range is used for highlights (such asspecular highlights) in HDR content and is typically mapped to thehighest brightnesses of HDR displays. However, SDR content does notinclude this highlight range and uses the upper portion of the SDR rangein a much more general purpose manner. If SDR content were mapped to theentire HDR range, then much of the SDR content would be in the highlightrange of the HDR display, which would be too bright. Thus instead ofmapping SDR content to the entire range of an HDR display, SDR contentis mapped to a much lower range. In an example, SDR content is mapped tothe brightness range of an SDR display, even when displayed on an HDRdisplay, so that the SDR content will appear on the HDR displayapproximately as it does on an SDR display.

There are issues with displaying content that includes a combination ofHDR and SDR content. More specifically, as described above, SDR contentis tone mapped such that the maximum brightness of the SDR content islower than the maximum brightness of both the HDR content and of thedisplay itself. This mapping can result in a situation in which an SDRoverlay becomes obscured (distorted or unviewable) due to being muchless bright than the underlying image content. Note, the SDR overlay, ifoverlaid on SDR content on an SDR display, would be much more visibledue to having comparable brightnesses to the SDR content. However, on anHDR display, the SDR overlay is overpowered by the HDR content, which isbrighter.

FIGS. 3 and 4A-4B illustrate a situation in which an SDR overlay ispresent over a content layer. FIG. 3 illustrates a screen 300 that hasan underlying content layer 305, for which one object 312 isillustrated. Although more objects may actually be present in thecontent layer, only one is shown for clarity of illustration. Inaddition, an overlay content layer 310 is illustrated. The overlaycontent layer 310 includes an overlay content layer 311 that at leastpartially overlaps with the underlying content layer object 312.

FIG. 4A illustrates, for an SDR display, comparative dynamic ranges ofthe display on which the screen 300 is shown, the underlying contentlayer 305, and the overlay content layer 310. Because the display is anSDR display, the display dynamic range 402, the underlying contentdynamic range 404, and the overlay dynamic range 406 are roughly thesame. The bright object 312 in the underlying content has a slightlylower brightness level than the overlay object 311 and thus the overlayobject 311 will likely not be obscured by the bright object 312 even ifthe overlay object 311 is displayed with alpha blending.

FIG. 4B illustrates, for an HDR display, comparative dynamic ranges ofthe display on which the screen 300 is shown, the underlying contentlayer 305, and the overlay content layer 310. Because the display is anHDR display, the display dynamic range 452 is much greater than the SDRdisplay dynamic range 402. Further, the underlying content provided tothe display is also HDR and has the same dynamic range as the display.However, because the overlay dynamic range is not HDR, but is insteadSDR, the overlay dynamic range is much lower than the display dynamicrange 452. Further, the maximum brightness of the overlay dynamic range456 is much lower than the maximum brightness of the display dynamicrange 452. Because the underlying content dynamic range 454 is HDR, thebright object 312 in the underlying content has a brightness that ismuch greater than that of the overlay object 311. For this reason, ifthe overlay object 311 is substantially transparent (e.g., has an alphavalue substantially lower than 1.0), then that overlay object 311 willbe obscured by the bright object 312 of the underlying content layer305. Note, the lines representing the overlay object 311 and the brightobject in the SDR display 400 diagram and the HDR display diagram 450represent the maximum brightness for those respective objects. It shouldbe understood that the dynamic range of the HDR content 454 does nothave to be exactly the same as the dynamic range of the display 452.

For the above reason, a technique is provided to adjust the brightnessvalues of at least a portion of a composited image that includes both anunderlying HDR content layer and an SDR overlay, where the compositedimage is to be displayed on an HDR display. Specifically, FIG. 5 is aflow diagram of a method 500 for adjusting a frame that includes HDRcontent with an SDR overlay, according to an example. Although describedwith respect to the system of FIGS. 1-3 and 4A-4B, it should beunderstood that any system that performs the steps of method 500 in anytechnically feasible order falls in the scope of the present disclosure.

The method 500 is performed by a compositor or by another unit differentthan the compositor, which may be a part of a processor 102 (e.g., as ahardware module or a software module executing on the processor 102), apart of the rendering engine 210 (e.g., as a post-processing hardware orsoftware module), a part of the pre-display device 215 (e.g., as ahardware or software module), or a part of the display device 118 (e.g.,again, as a hardware or software module). A tone mapper first processescontent to map the brightnesses defined in the content to brightnessesof a display. As described above, SDR content is mapped to an SDRdynamic range and HDR content is mapped to a greater, HDR dynamic range.When compositing is required, such as in the situation an SDR overlay isto be displayed over HDR content, a compositor blends the SDR and HDRcontent according to the alpha values of the SDR overlay. The compositormay be a software or hardware component located in any of the unitsdescribed above or in another unit not described. To overcome thedifficulties associated with an SDR overlay having lower maximumbrightness than HDR underlying content, the compositor performs themethod 500 when compositing the HDR and SDR content. The compositoroptionally analyzes one or more characteristic values of pixels of inputcontent, (where input content includes underlying content as well asoverlay content), and (in at least some situations) adjusts thebrightnesses of one or more pixels of one or both of the underlyingcontent and the overlay content such that, when blended, the overlaycontent has a different appearance than if no adjustments were made andthe underlying content and overlay content were blended. Thismodification helps to improve contrast between SDR overlay and HDRunderlying content so that the SDR overlay is more visible to a viewer.The term “characteristic value” refers to any value that characterizes apixel. Examples of characteristic values include alpha values, luminancevalues, average color values (or other functions of color values), orany other value that characterizes a pixel.

The method 500 begins at step 502, where the compositor determines thatan image to be displayed on the display device 118 includes HDRunderlying content and an SDR overlay. Examples of situations includingHDR underlying content and an SDR overlay include a renderedthree-dimensional image (the underlying content) that is rendered in HDRand an SDR overlay generated by, for example, one or more of theprocessor 102 at the request of an application, operating system, devicedriver, or other hardware or software module, the rendering engine 210,or the pre-display device 215, HDR television content delivered over acable television network where a set-top box provides SDR menus to bedisplayed over the HDR television content, or a media player (e.g.,blu-ray player) that displays HDR content from a physical media and alsodisplays SDR menus, text, and/or graphics over the HDR content from thephysical media. Although these examples are described, these examplesshould not be taken to be limiting.

At step 504, the compositor identifies areas of overlap between the HDRunderlying content and the SDR overlay. These areas include pixels thatare covered by both the SDR overlay and the HDR underlying content. Ifany portion of the SDR overlay is not completely opaque (e.g., has analpha value of less than 1.0), then that SDR overlay will be displayedwith at least some contribution from the underlying HDR content.

At step 506, the compositor determines one or more adjustments to applyto either or both of the SDR overlay and the HDR underlying content. Alarge variety of adjustments are possible and will be discussed infurther detail below. At step 508, the compositor applies the one ormore adjustments to either or both of the overlay and the underlyingcontent. After the method 500, the SDR overlay and HDR underlyingcontent are composited to form a final image, which is then displayed onthe display 118.

Example adjustments to be applied by the compositor are now described.In one example, the compositor applies a fixed adjustment to pixels ofone or more of the HDR underlying content and the SDR overlay. In someexamples, this fixed adjustment includes one or more of adding orsubtracting a fixed value to the luminance of the HDR underlying contentpixels, multiplying the luminance of the HDR underlying content pixelsby a fixed value (e.g., either a reducing value like a value between 0and 1 or an increasing value like a value greater than 1), adding orsubtracting a fixed value to the luminance of the SDR overlay,multiplying the luminance of the SDR overlay by a fixed value (e.g.,either a reducing value like a value between 0 and 1 or an increasingvalue like a value that is greater than 1), and clipping a luminance ofthe HDR layer. One or more of the above adjustments may be applied aloneor in combination. Clipping is an adjustment that sets a maximumluminance value for the underlying HDR content. To apply a clippingadjustment, if the luminance value of a pixel is above the maximum, theluminance value is set to the maximum value, and no clipping adjustmentis applied if the luminance value of the pixel is below or equal to themaximum. In some examples, other adjustments (e.g., multiplicationand/or addition) are first applied to luminance values of pixels andthen the resulting value is clipped to a maximum value if necessary.Clipping may also be used alone, without any other adjustments. In oneexample, all pixels in an overlap area (the pixels in the area ofoverlap between the HDR content and the SDR content) have theirluminances reduced through either or both of multiplication andaddition. The resulting values that are over the clipping maximum arethen reduced further, either through being set to the maximum value orthrough a further luminance value reduction (e.g., by multiplying from avalue between 0 and 1), followed by a clipping operation, wherein valuesthat remain over the clipping maximum are set to the maximum value.

In some implementations, clipping is applied in a more complex way. Morespecifically, clipping maximums can be set for red, green, and bluecomponents of pixels, and applied independently. In someimplementations, only one or two of the red, green, or blue componentsare clipped. Any other technically feasible technique for clipping pixelvalues may be used.

In some examples, a fixed adjustment includes adding or subtracting avalue and the multiplying the result by a scale value. In some examples,an addition or subtraction operation and then a multiplication operationis applied to both the SDR content and the HDR content.

In one example, the fixed adjustment is based on a comparison betweenthe dynamic ranges of the SDR overlay and the HDR underlying content,with a greater disparity resulting in stronger modifications toluminance. In another example, the fixed adjustment is a pre-definedconstant value. In another example, the fixed adjustment isuser-defined. The manner in which the value of the fixed adjustment isdecided should not be read to be limited by the specific examplesprovided herein.

In another example, the compositor applies adjustments that are contentadaptive. Content adaptive adjustments are also referred to as variableadjustments herein. In general, content adaptive adjustments make one ormore modifications to luminance values of pixels of either or both ofthe HDR underlying content and the SDR overlay, where the degree towhich the luminance is adjusted is based on an analysis ofcharacteristic values of pixels of either or both of the HDR underlyingcontent and the SDR overlay. Some examples of content adaptiveadjustments follow.

In an example of a content adaptive adjustment, the compositor applies avariable adjustment to pixels of one or more of the HDR underlyingcontent and the SDR overlay. In some examples, the variable adjustmentincludes one or more of adding or subtracting a variable value from theluminance of the HDR underlying content pixels, multiplying theluminance of the HDR underlying content pixels by a variable value(e.g., a reducing value such as a value between 0 and 1 or an increasingvalue such as a value greater than 1), adding or subtracting a variablevalue to the luminance of the SDR overlay, or multiplying the luminanceof the SDR overlay by a variable value (e.g., a reducing value such as avalue between 0 and 1 or an increasing value such as a value that isgreater than 1). In any of the above examples, the variable value isbased on a characteristic value of one or more pixels of the HDRunderlying content and the SDR overlay. Variable adjustments are contentadaptive in that the adjustment to be applied is based on some aspect ofthe pixels of either or both of the SDR overlay or the HDR underlyingcontent.

The above adjustments are referred to in other portions of thisdisclosure. Adjustments to the SDR overlay pixels that are fixed arereferred to collectively as “fixed SDR adjustments.” The term “fixed”means that the adjustment does not vary for different pixels in acertain region of the screen (such as the whole screen, a smallerportion of the whole screen, or in the region of overlap between theoverlay and the underlying content). These adjustments include the fixedadjustments to the SDR overlay mentioned above. Adjustments to the HDRunderlying content that are fixed are referred to collectively as “fixedHDR adjustments.” These adjustments include the fixed adjustments to theHDR underlying content mentioned above. Adjustments to SDR content thatare variable are referred to herein as “variable SDR adjustments.” Theseadjustments include the variable adjustments to the SDR overlaymentioned above. Adjustments to the HDR underlying content that arevariable are referred to collectively as “variable HDR adjustments.”These adjustments include the variable adjustments to the HDR underlyingcontent mentioned above. The term “fixed adjustments” referscollectively to fixed SDR adjustments and fixed HDR adjustments. Theterm “variable adjustments” refers collectively to variable SDRadjustments and variable HDR adjustments. The term “adjustments” herein,when used without any other modifier, refers collectively to both fixedand variable adjustments. In the present disclosure, the term “weakeradjustment” is an adjustment that affects some characteristic of a pixelto a lesser degree than a “stronger adjustment.”

In some implementations, adjustments are applied to the pixels for whichthere is overlap between the SDR overlay and the underlying HDR contentand not to pixels for which no such overlap exists. In some examples, avariable adjustment to a particular pixel is based on the characteristicvalue (or values) of the pixel being adjusted pixel and not on otherpixels. In an example, a pixel of the SDR overlay having hightransparency (i.e., alpha close to 0) would receive a “strongeradjustment” (e.g., a greater boost to luminance) than a pixel having alow transparency (which would receive a weaker adjustment, such as alower boost to luminance). In other examples, adjustments applied topixels are based on an analysis of multiple pixels, which are either allfrom one layer (e.g., all from the SDR layer or all from the HDR layer)or are from both layers.

The compositor applies fixed adjustments as follows. First, thecompositor determines whether a particular pixel is to receive anadjustment. Then the compositor applies a fixed adjustment to that pixelif it is to receive an adjustment. In one example, a determination ofwhether to apply an adjustment to a pixel occurs by determining whetherthe characteristic value of the pixel is above or below a threshold. Inone example, if an SDR overlay pixel has an alpha value lower than aparticular threshold, then the compositor applies a fixed adjustment tothat SDR overlay pixel (e.g., by increasing the luminance of that pixelthrough addition of a fixed value or multiplication by a fixed gain),applies a fixed adjustment to the overlapped pixel of the underlying HDRcontent layer (e.g., by decreasing the luminance of that pixel bysubtracting a fixed value or multiplying by a fixed value that is lessthan 0) or applies both of these adjustments to the SDR overlay pixeland the HDR content layer pixel.

For variable adjustments for which the adjustment made to any particularpixel is based on an analysis of characteristic values of multiplepixels, the compositor uses a statistical technique to determine whetherand to what degree a particular pixel is to receive an adjustment. Thestatistical technique takes into consideration characteristic values ofpixels for which there is overlap between the SDR overlay and the HDRunderlying content and produces a statistical result based on thosecharacteristic values. In some implementations, the compositor comparesthe statistical result to a threshold. If the statistical result isgreater than or lower than the threshold, then the compositor appliesadjustments to pixels of either or both of the SDR overlay and the HDRunderlying content. In addition, in some implementations, the strengthof an adjustment made to a pixel is based on a comparison between thecharacteristic value of that pixel and the statistical result.

In an example of the above, adjustments are made to pixels of the SDRoverlay but not to pixels of the HDR underlying content. The compositorcalculates a mean of all of the alpha values of the pixels of the SDRoverlay and determines that the mean is below a threshold (indicatingthere is significant transparency). In response to this determination,the compositor determines that adjustments should be applied to thepixels of the SDR overlay. The adjustments made are based on acomparison between the alpha values of the SDR pixels and the calculatedaverage alpha value. For pixels that have a lower alpha than the averagevalue, a greater boost to luminance is applied. For pixels that have agreater alpha than the average value, a lower boost to luminance isapplied.

In an example, the compositor determines which adjustments to apply towhich pixels based on a more complex statistical technique such as ahistogram. According to such a technique, the compositor generates ahistogram based on characteristic values for pixels of the SDR overlay,overlapped pixels of the HDR underlying content, or both. The histogramgroups characteristic values of pixels into histogram bins thatcorrespond to ranges of characteristic values. Each bin stores thenumber of values that fall within the range assigned to that bin.

The compositor determines whether and, optionally, to what degree toapply adjustments to pixels of the SDR overlay and/or the HDR underlyingcontent based on the histogram. In one example, a thresholdcharacteristic value exists and the compositor determines the number ofpixels for which the characteristic value is above (or below) thethreshold using the histogram. For example, if the count for histogrambins above the threshold value is above a count threshold, then thecompositor applies an adjustment to one or more pixels of the SDRoverlay and/or the HDR underlying content. In an example, the compositorapplies variable adjustments to one or more pixels of the SDR overlayand/or the HDR underlying content, where the adjustment to be applied isbased on the distribution defined by the histogram. For a histogramhaving the majority of pixels near the opaque end (e.g., more than athreshold percentage of pixels have an alpha value above an alpha valuethreshold), the adjustment applied is weaker than for a histogram havingthe majority of pixels near the transparent end.

In one example, the histogram is a histogram of luminance values for theHDR underlying content. If the histogram indicates that the underlyingcontent is brighter, on average, than a threshold (e.g., a number ofpixels for bins above the threshold is greater than a count threshold),then the compositor determines that adjustments should be applied. In anexample, the adjustments are stronger if the histogram indicates thatthe underlying HDR content is brighter and are weaker if the histogramindicates that the underlying HDR content is dimmer.

In an example, the compositor performs steps 506 and 508 in multiplestages. Specifically, each stage is associated with a different spatialpixel scope. In one stage, the compositor determines characteristicvalues based on one spatial pixel scope and applies adjustments topixels based on an analysis of those characteristic values. Then inanother stage, the compositor determines characteristic values based ona different spatial pixel scope and applies adjustments to the pixelsbased on an analysis of those characteristic values. Additional stages,where adjustments based on analysis of different spatial pixel scopesare applied, may be performed as well. In some examples, differentspatial pixel scopes include global, local, and single pixel. A globalpixel scope is one in which pixels from approximately the entire extentsof the image are considered in generating characteristic values foranalysis. Adjustments are made to certain pixels based on this analysis.For example, an average luminosity of pixels across the entire image maybe taken. A local pixel scope is a scope in which pixels around thepixels to which adjustments are made are considered in generatingcharacteristic values for analysis. A single pixel scope is a scope inwhich characteristic values are derived from the pixel to whichadjustments are made. In examples, the compositor applies adjustments topixels based on characteristic values for one spatial pixel scope, thenapplies adjustments to pixels based on characteristic values for adifferent pixel scope.

It should be understood that many variations are possible based on thedisclosure herein. Although features and elements are described above inparticular combinations, each feature or element can be used alonewithout the other features and elements or in various combinations withor without other features and elements.

The methods provided can be implemented in a general purpose computer, aprocessor, or a processor core. Suitable processors include, by way ofexample, a general purpose processor, a special purpose processor, aconventional processor, a digital signal processor (DSP), a plurality ofmicroprocessors, one or more microprocessors in association with a DSPcore, a controller, a microcontroller, Application Specific IntegratedCircuits (ASICs), Field Programmable Gate Arrays (FPGAs) circuits, anyother type of integrated circuit (IC), and/or a state machine. Suchprocessors can be manufactured by configuring a manufacturing processusing the results of processed hardware description language (HDL)instructions and other intermediary data including netlists (suchinstructions capable of being stored on a computer readable media). Theresults of such processing can be maskworks that are then used in asemiconductor manufacturing process to manufacture a processor whichimplements aspects of the embodiments.

The methods or flow charts provided herein can be implemented in acomputer program, software, or firmware incorporated in a non-transitorycomputer-readable storage medium for execution by a general purposecomputer or a processor. Examples of non-transitory computer-readablestorage mediums include a read only memory (ROM), a random access memory(RAM), a register, cache memory, semiconductor memory devices, magneticmedia such as internal hard disks and removable disks, magneto-opticalmedia, and optical media such as CD-ROM disks, and digital versatiledisks (DVDs).

What is claimed is:
 1. A method for adjusting an image including a firstrange underlying content layer and a second range overlay, wherein thesecond range is less than the first range the method comprising:determining that the image includes both the first range underlyingcontent layer and the second range overlay; identifying pixels for whichoverlap exists between the first range underlying content layer and thesecond overlay; determining one or more adjustments to make to pixels ofeither or both of the first range underlying content layer and thesecond range overlay; and applying the one or more adjustments to thepixels.
 2. The method of claim 1, wherein the one or more adjustmentscomprise fixed adjustments that are the same for each pixel.
 3. Themethod of claim 1, wherein the one or more adjustments compriseadjustments to luminance of the pixels.
 4. The method of claim 1,wherein the one or more adjustments comprise one of multiplication ofthe pixel by a value, clipping the pixel, or an addition or subtractionof a value to or from the pixels.
 5. The method of claim 1, wherein theone or more adjustments comprise variable adjustments that are differentfor different pixels.
 6. The method of claim 5, wherein the value of avariable adjustment is based on a statistical analysis of the pixels. 7.The method of claim 6, wherein the statistical analysis comprisescalculating an average of the pixels.
 8. The method of claim 6, whereinthe statistical analysis comprises constructing a histogram.
 9. Themethod of claim 6, wherein determining the one or more adjustments tomake comprises determining whether to make an adjustment to a pixel bycomparing a characteristic value of the pixel to a statistical resultthat is based on the statistical analysis.
 10. A device for adjusting animage including a first range underlying content layer and a secondrange overlay, wherein the second range is less than the first range,the device comprising: a compositor configured to: determine that theimage includes both the first range underlying content layer and thesecond range overlay; identify pixels for which overlap exists betweenthe first range underlying content layer and the second range overlay;determine one or more adjustments to make to pixels of either or both ofthe first range underlying content layer and the second range overlay;apply the one or more adjustments to the pixels; and output the adjustedpixels to a display device for display.
 11. The device of claim 10,wherein the one or more adjustments comprise fixed adjustments that arethe same for each pixel.
 12. The device of claim 10, wherein the one ormore adjustments comprise adjustments to luminance of the pixels. 13.The device of claim 10, wherein the one or more adjustments comprise oneof multiplication of the pixel by a value, clipping the pixel, or anaddition or subtraction of a value to or from the pixels.
 14. The deviceof claim 10, wherein the one or more adjustments comprise variableadjustments that are different for different pixels.
 15. The device ofclaim 14, wherein the value of a variable adjustment is based on astatistical analysis of the pixels.
 16. The device of claim 15, whereinthe statistical analysis comprises calculating an average of the pixels.17. The device of claim 15, wherein the statistical analysis comprisesconstructing a histogram.
 18. The device of claim 15, whereindetermining the one or more adjustments to make comprises determiningwhether to make an adjustment to a pixel by comparing a characteristicvalue of the pixel to a statistical result that is based on thestatistical analysis.
 19. The device of claim 10, wherein the compositorcomprises a compositor circuit.
 20. A non-transitory computer-readablemedium storing instructions that, when executed by a processor, causethe processor to adjusting an image including a first range underlyingcontent layer and a standard dynamic range second range overlay, whereinthe second range is less than the first range, by: determining that theimage includes both the first range underlying content layer and thesecond range overlay; identifying pixels for which overlap existsbetween the first range underlying content layer and the second rangeoverlay; determining one or more adjustments to make to pixels of eitheror both of the first range underlying content layer and the second rangeoverlay; and applying the one or more adjustments to the pixels.
 21. Thenon-transitory computer-readable medium of claim 20, wherein the one ormore adjustments include one or more of a fixed adjustment or a variableadjustment.